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length  of  commitment  required  has  altered  as  the  needs  of  the  Air  Force  have  changed. 
The  Air  Force  believes  that  increasing  the  length  of  commitment  will  help  to  address 
long-term  pilot  supply  issues.  This  thesis  takes  an  empirical  look  at  one  possible 
negative  side  effect  of  increasing  the  length  of  commitment  owed,  namely  the  possibility 
of  varying  effects  on  the  probability  of  attending  pilot  training  on  individuals  of  different 
ability  levels,  demographic  make-ups,  information  positions,  and  macroeconomic 
conditions.  The  group  used  for  studying  these  issues  is  a  collection  of  graduates  from  the 
United  States  Air  Force  Academy  from  the  class  years  of  1985  through  2000.  Using  this 
dataset,  which  has  a  plethora  of  performance  and  demographic  data,  a  linear  probability 
model  is  estimated  to  determine  the  effect  on  the  probability  of  selecting  pilot  training 
due  to  an  increased  commitment  length.  The  results  of  this  study  show  that  increasing 
the  length  of  commitment  leads  to  a  negative  impact  on  the  probability  of  attending  pilot 
training  for  nearly  all  of  the  members  of  the  dataset.  Furthermore,  the  effects  are  stronger 


on  the  individuals  with  higher  performance,  as  measured  by  several  different  measures 
used  by  the  Air  Force  Academy.  In  multiple  interaction  models,  the  negative  effects  are 
stronger  on  individuals  showing  improvement  in  performance  versus  individuals  with 
declining  performance.  There  are  no  differences  in  the  effects  on  groups  with  different 
demographics  and  information  positions.  Additionally,  the  macroeconomic  conditions 


have  small  bearings  on  the  results. 
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This  thesis  explores  the  effects  of  changing  the  length  of  service  owed  to  the  Air 
Force  for  attending  pilot  training.  The  Air  Force  demands  service  as  repayment  for  the 
expense  incurred  by  the  government  to  provide  the  training.  From  time  to  time,  the 
length  of  commitment  required  has  altered  as  the  needs  of  the  Air  Force  have  changed. 
The  Air  Force  believes  that  increasing  the  length  of  commitment  will  help  to  address 
long-term  pilot  supply  issues.  This  thesis  takes  an  empirical  look  at  one  possible 
negative  side  effect  of  increasing  the  length  of  commitment  owed,  namely  the  possibility 
of  varying  effects  on  the  probability  of  attending  pilot  training  on  individuals  of  different 
ability  levels,  demographic  make-ups,  information  positions,  and  macroeconomic 
conditions.  The  group  used  for  studying  these  issues  is  a  collection  of  graduates  from  the 
United  States  Air  Force  Academy  from  the  class  years  of  1985  through  2000.  Using  this 
dataset,  which  has  a  plethora  of  performance  and  demographic  data,  a  linear  probability 
model  is  estimated  to  determine  the  effect  on  the  probability  of  selecting  pilot  training 
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due  to  an  increased  commitment  length.  The  results  of  this  study  show  that  increasing 
the  length  of  commitment  leads  to  a  negative  impact  on  the  probability  of  attending  pilot 
training  for  nearly  all  of  the  members  of  the  dataset.  Furthermore,  the  effects  are  stronger 
on  the  individuals  with  higher  performance,  as  measured  by  several  different  measures 
used  by  the  Air  Force  Academy.  In  multiple  interaction  models,  the  negative  effects  are 
stronger  on  individuals  showing  improvement  in  performance  versus  individuals  with 
declining  performance.  There  are  no  differences  in  the  effects  on  groups  with  different 
demographics  and  information  positions.  Additionally,  the  macroeconomic  conditions 
have  small  bearings  on  the  results. 
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INTRODUCTION 


In  the  fall  of  1998,  the  Air  Force  announced  that  the  length  of  the  service 
commitment  owed  for  the  completion  of  pilot  training  was  being  increased  to  ten  years, 
effective  October  1 , 1 999.  According  to  General  Michael  Ryan,  Air  Force  Chief  of  Staff, 
this  change  was  made  to  “ensure  we  meet  our  Air  Force  pilot  needs  in  the  future” 
(“Active  Duty”,  1998).  The  Air  Force’s  goal  is  to  get  more  of  a  return,  in  the  form  of 
additional  service  time,  on  its  considerable  investment  in  its  pilots. 

Training  members  of  the  military  for  specific  jobs  can  be  extremely  costly  due  to 
the  highly  technical  nature  of  the  training  required  to  gain  proficiency  in  many  military 
jobs.  This  is  particularly  true  in  the  case  of  training  pilots  for  the  U.S.  Air  Force.  The 
average  investment  in  a  pilot  over  the  first  nine  years  of  service  is  $5.9  million  (“Active 
Duty”,  1998).  This  money  is  not  spent  without  receiving  something  in  exchange  from 
those  who  get  the  training.  Depending  on  the  type  of  training  received,  service  members 
agree  to  serve  a  length  of  time  in  the  military  to  “pay”  the  government  back  for  the 
expense  of  the  training.  This  commitment  to  serve  is  known  as  the  Active  Duty  Service 
Commitment,  or  ADSC.  As  mentioned  above,  the  ADSC  for  the  completion  of  pilot 
training  is  currently  ten  years. 

This  commitment  length  has  varied  over  time  based  on  the  manpower  needs  of 
the  Air  Force.  The  10-year  commitment  took  effect  on  October  1,  1999,  and  applies  to 
all  officers  commissioned  in  2000  and  beyond  (“Active  Duty”,  1998).  During  the  period 
of  1988  until  1999,  the  commitment  for  pilot  training  was  8  years.  The  commitment 
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during  1987  was  7  years.  Prior  to  1987,  only  6  years  of  active  duty  was  required  to  pay 
back  the  Air  Force  (Duffet,  2001).  The  stated  purpose  for  the  most  recent  change  was  to 
create  a  long-term  solution  to  the  current  pilot  shortage,  caused  by  low  retention  rates  and 
limited  training  capacity.  The  Air  Force  leadership  felt  that  the  increase  in  the  length  of 
ADSC  for  pilot  training  would  help  to  alleviate  this  shortage. 

While  the  Air  Force  apparently  believes  the  policy  change  would  lead  to  an 
improvement  in  the  state  of  the  pilot  force,  this  may  not  necessarily  be  the  case.  The 
potential  exists  for  negative  impacts  on  individuals’  decisions  to  attend  pilot  training 
caused  by  this  policy  change.  Two  possible  such  impacts  might  be  caused  by  the  policy. 
First,  the  number  of  pilots  electing  to  attend  pilot  training  might  decline  as  the 
opportunity  cost  of  pilot  training  has  increased.  Second,  the  policy  change  could  also 
affect  the  quality  of  those  candidates  who  elect  to  go  to  pilot  training.  If  the  increase  in 
opportunity  cost  causes  a  change  in  behavior,  it  will  do  so  in  those  candidates  who  have 
the  most  potential  to  earn  more  elsewhere.  There  exists  the  possibility  that  this  could 
reduce  the  overall  quality  of  the  pilot  corps  by  encouraging  the  best  candidates  to  alter 
their  career-path  selection. 

This  thesis  focuses  on  the  second  issue  described  above,  namely  the  issue 
regarding  the  quality  of  those  candidates  who  elect  to  attend  pilot  training.  The  overall 
goal  of  this  study  is  to  determine  if  the  policy  has  any  negative  impacts  on  the  quality  of 
the  candidates  who  attend  pilot  training.  A  variety  of  data  is  collected  and  analyzed  to 
determine  the  effect  that  the  length  of  time  owed  for  pilot  training  has  on  the  decision  to 
enter  pilot  training.  Additionally,  the  data  is  studied  to  see  if  the  impact  of  the  policy 
varies  with  the  quality  of  the  potential  candidate. 


3 


This  thesis  first  describes  the  system  for  becoming  a  pilot  in  the  Air  Force.  The 
data  is  discussed,  elaborating  on  both  the  sources  and  the  meaning  of  the  data  elements. 
After  this,  the  model  used  to  analyze  the  data  is  explained  and  the  results  from  this  model 
are  detailed.  The  thesis  then  describes  several  alternate  studies  done  to  confirm  the 
validity  of  some  of  the  procedures  used.  Potential  points  of  contention  with  the  study  are 
raised  and  discussed  in  the  following  section.  The  thesis  then  concludes  with  a  final 
discussion  of  the  meaning  of  the  results  and  some  recommendations  based  on  these 
results. 


PILOT  TRAINING  IN  THE  AIR  FORCE 
Before  discussing  the  method  of  analysis  used  to  determine  the  impact  of  the 

policy,  it  is  worthwhile  to  explain  the  methods  of  getting  to  pilot  training  in  the  Air 
Force.  To  become  a  pilot  in  the  Air  Force,  a  person  must  first  receive  a  commission  as  an 
officer.  There  are  three  different  pathways  for  obtaining  a  commission  and  becoming  a 
potential  pilot  candidate.  These  three  commissioning  sources  are  Reserve  Officer 
Training  Corps,  Officer  Training  School,  and  the  Air  Force  Academy.  Each  source  is 
different  from  the  others,  but  all  yield  the  same  end  result  of  a  commission  in  the  Air 
Force. 

Reserve  Officer  Training  Corps  (ROTC)1  is  a  geographically  diverse  program, 
with  detachments  at  civilian  universities  throughout  the  country.  Air  Force  ROTC  is  a 
program  designed  to  commission  officers  while  they  complete  their  college  degrees. 
While  enrolled  in  ROTC,  an  individual  receives  military  training  and  attends  a  civilian 
college  at  the  same  time.  Cadets  enrolled  in  ROTC  can  also  qualify  for  scholarships  to 
cover  the  costs  of  tuition.  The  program  can  last  anywhere  from  one  to  four  years, 
depending  on  the  needs  of  the  Air  Force  for  personnel.  During  each  year,  cadets  take 
classes  that  teach  basic  Air  Force  knowledge  and  professional  officer  skills.  During  one 
summer  of  the  program,  cadets  attend  a  four-  to  six- week  field-training  course,  where 

1  Information  on  ROTC  is  taken  from  the  ROTC  website, 

http://afoats.maxwell.af.mil/rotc.htm.  and  from  the  University  of  Florida  Air  Force 
ROTC  website,  www . afrotc .ufl . edu . 
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they  receive  weapons  training  and  survival  training,  among  other  things.  Upon  the 
completion  of  the  ROTC  program,  cadets  receive  a  commission  in  the  Air  Force  as  a 
second  lieutenant. 

Officer  Training  School  (OTS)2  is  a  twelve-week  course  to  commission  officers 
into  the  Air  Force.  It  consists  of  classroom  work,  lectures,  and  exercises  designed  to 
build  leadership  and  develop  the  skills  needed  by  an  officer.  It  is  only  open  to  college 
graduates  under  the  age  of  35.  Upon  completion  of  the  program,  graduates  are 
commissioned  into  the  Air  Force. 

The  United  States  Air  Force  Academy  (USAFA)  is  a  four-year  institution  where 
cadets  earn  a  Bachelor  of  Science  degree  in  addition  to  a  commission  in  the  Air  Force. 
The  Air  Force  Academy  differs  from  ROTC  in  that  it  is  a  military  university  with  a  large 
portion  of  the  faculty  being  officers.  ROTC  is  a  class  at  a  regular  university  where  the 
only  instructors  who  are  officers  are  those  teaching  the  ROTC  program. 

The  three  commissioning  programs  are  similar  in  that  they  all  teach  the  same 
basics  of  officership  and  Air  Force  knowledge.  Graduates  of  any  of  the  three  programs 
receive  a  commission  into  the  Air  Force  upon  completion  of  the  program.  The  primary 
differences  are  in  the  location  of  training  and  the  point  in  a  person’s  life  at  which  he  or 
she  enters  the  training.  Once  the  commission  is  received,  there  is  no  difference  due  to  the 
source  of  the  commission;  a  lieutenant  is  a  lieutenant,  regardless  of  the  commissioning 
source. 


2  Information  on  OTS  is  taken  from  the  Air  Force  OTS  website, 
http://ots.maxwell.af.mil/. 
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Just  prior  to  the  completion  of  the  commissioning  program,  the  cadets  undergo  a 
process  to  determine  which  career  field  they  will  enter  in  the  Air  Force.  For  reasons  that 
will  be  explained  later,  this  thesis  examines  data  on  graduates  from  USAFA  only. 

Because  of  this  selection  of  a  limited  pool  of  candidates  for  study,  only  the  process  of 
selecting  a  career  field  for  USAFA  cadets  will  be  explained. 

Before  being  admitted  to  the  Academy,  candidates  must  undergo  a  fairly  rigorous 
medical  examination  to  determine  their  medical  status  for  service.  This  is  the  first  step  in 
becoming  a  pilot,  as  even  stricter  medical  standards  are  required  of  pilots  than  of  non¬ 
flying  officers. 

While  at  the  Academy,  cadets  are  evaluated  in  many  different  areas,  including 
academic  performance  in  a  set  of  core  classes  common  to  all  cadets,  academic 
performance  within  the  cadets’  selected  majors,  military  performance,  and  athletic 
performance.  Each  of  these  areas  is  used  in  determining  the  cadet’s  rank,  or  order  of 
merit,  in  the  class. 

Midway  through  the  junior  year  of  the  Academy,  cadets  undergo  another  round  of 
medical  screenings  to  determine  their  final  medical  status  at  graduation.  At  the  beginning 
of  the  senior  year,  cadets  fill  out  a  listing  of  the  Air  Force  jobs,  known  as  Air  Force 
Specialty  Codes  or  AFSCs,  which  they  would  like  to  have  after  graduation.  First,  cadets 
who  are  medically  qualified  are  asked  if  they  want  to  fly.  Then,  all  cadets,  including 
those  who  want  to  fly,  are  required  to  list  their  top  five  non- flying  jobs.  At  the  beginning 
of  the  final  semester  at  the  Academy,  cadets  finally  pick  their  actual  jobs.  First,  cadets 
are  split  into  two  groups:  those  seeking  pilot  assignments  and  those  not  seeking  pilot 
assignments.  The  selection  then  goes  by  the  order  of  merit.  The  first  person  in  the  order 
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of  merit  for  pilot-qualified  cadets  selects  the  pilot  slot  that  that  person  desires.  The 
selection  continues  in  order  of  merit  until  all  of  the  pilot  slots  are  gone  or  until  all  who 
are  qualified  for  one  and  want  one  have  selected  one.  The  remaining  people  fall  into  the 
category  of  those  who  did  not  want  a  pilot  slot,  were  not  medically  qualified  for  a  pilot 
slot,  or  did  not  have  the  opportunity  to  pick  one  because  all  available  slots  were  taken. 
These  cadets  then  pick  the  non-flying  jobs  in  a  similar  fashion  to  the  process  of  picking 
pilot  slots.  Upon  graduation,  the  newly  commissioned  lieutenants  move  on  to  their  new 
jobs  for  a  minimum  of  5  years,  the  ADSC  for  the  Academy  education,  unless  their  job 
requires  additional  training,  and  thus  an  additional  ADSC,  such  as  the  ten-year 
commitment  for  pilot  training. 


DATA 


As  mentioned  above,  only  data  from  USAFA  cadets  are  used  in  this  study.  There 
are  several  reasons  for  using  data  from  this  source  only.  First  of  all,  while  USAFA 
graduates  account  for  only  19.5%  of  commissioned  officers  in  the  Air  Force,  they 
represent  44.4%  of  pilots.  This  makes  USAFA  the  single  largest  source  of  pilots  for  the 
Air  Force.  The  next  largest  source  of  pilots  is  ROTC,  providing  42.9%  of  current  pilots 
(Air  Force  Personnel  Center,  2001).  However,  since  the  ROTC  detachments  are  at 
different  universities  spread  across  the  country  with  little  interaction  between 
detachments,  it  is  hard  to  view  ROTC  as  a  single  source  where  each  officer  receives  as 
similar  an  experience  as  all  Academy  graduates  share.  While  the  core  of  the  experience 
and  requirements  for  the  ROTC  program  are  the  same,  the  individual  culture  and  college 
experiences  differ  with  each  university.  Each  individual  ROTC  detachment  provides 
only  a  small  portion  of  the  pilots  for  the  Air  Force.  In  contrast,  USAFA  provides  a 
source  of  common  experience  that  can  eliminate  many  other  unobserved  effects  in  the 
study. 

In  addition  to  being  the  largest  single  source  of  pilots  for  the  Air  Force,  USAFA 
presents  several  other  advantages  for  research.  Unlike  OTS,  it  is  a  four-year  program, 
which  provides  more  opportunity  for  assessment  of  the  cadets  before  they  make  their 
decision  regarding  their  Air  Force  job.  Additionally,  USAFA  collects  and  maintains  a 
wide  variety  of  data  on  its  students’  pre-college  preparation  and  performance  at  the 
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Academy,  providing  an  easy  source  for  uniform  and  viable  data.  A  more  robust  study 
would  attempt  to  integrate  data  from  the  other  commissioning  sources  into  the  study. 

For  this  study,  a  model  is  constructed  to  determine  the  impact  of  the  policy 
regarding  commitment  length.  Before  explaining  the  model  used,  the  covariates  used  to 
create  the  model  and  the  sources  of  these  covariates  are  described. 

The  principal  source  of  data  for  this  study  is  the  database  of  information  on 
graduates  of  USAFA  maintained  by  the  Office  of  Institutional  Research  and  Assessment 
(XPR).  XPR  is  the  office  in  charge  of  research  for  USAFA.  Its  job  is  providing  “studies 
and  analysis  services  for  policy  and  decision  making,  coordinating  assessment  efforts 
across  Academy  organizations,  managing  institutional  databases,  and  providing  statistical 
data  for  internal  and  external  customers”  (Office  of  Institutional  Research  and 
Assessment  website,  2001).  XPR  maintains  data  on  a  cadet’s  demographics, 
performance,  and  discipline  history,  among  other  things.  All  out-processing  and 
graduating  cadets  complete  several  surveys,  with  the  results  of  those  surveys  being 
incorporated  into  the  database.  XPR’s  database  provides  the  largest  source  of 
demographic  and  performance  data  on  individual  graduates.  In  order  to  obtain 
information  on  post-USAFA  career  choices,  these  data  are  merged  with  a  file  from  the 
Air  Force  Personnel  Center,  which  provides  information  on  the  status  of  graduates  of 
USAFA  regarding  pilot  training. 

Some  of  the  covariates  used  are  fairly  common  data  elements  and  will  not  be 
explained  in  detail.  These  elements  include  race,1  sex,  the  state  of  residence  prior  to 

1  The  only  categories  for  race  are  Native  American,  Asian/Pacific  Islander,  Black, 
Hispanic,  White,  and  Other.  Dummy  variables  are  created  to  allow  each  cadet  to  be 
classified  in  one  of  these  racial  categories. 
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entering  USAFA,  the  academic  major  at  USAFA,  SAT  math  and  verbal  scores,  SAT2 
scores,  and  ACT  math,  verbal,  science,  and  reading  scores.  Other  covariates  are  more 
specific  to  the  Academy  situation  or  have  some  unique  properties  and  thus  warrant  some 
explanation.  These  covariates  include  data  to  determine  admission  to  the  Academy, 
performance  data  while  at  the  Academy,  and  data  regarding  the  parental  military  status, 
medical  status  and  graduation  status  of  the  cadet.  Table  1  presents  some  summary 
statistics  on  the  numerical  data. 


Table  1.  Summary  Statistics  on  Numerical  Data. 


Variable 

Observations 

Mean 

Standard  Deviation 

SAT  Math 

4405 

662.8218 

56.26351 

SAT  Verbal 

4405 

577.1305 

62.85829 

ACT  Math 

3892 

29.77569 

3.186613 

ACT  Verbal 

3892 

25.57734 

3.193081 

SAT2  Reading 

330 

533.4242 

78.50872 

SAT2  Science 

331 

676.5559 

67.7221 1 

ACT  Reading 

3892 

27.89106 

3.922594 

ACT  Science 

3892 

30.16675 

2.917295 

Weighted  Composite 

8297 

40.81145 

Academic  Composite 

8297 

261 .8226 

Graduation  Order  of  Merit3 

8297 

28.44962 

Academic  Order  of  Merit3 

8297 

50.0915 

28.61418 

Military  Order  of  Merit3 

8297 

51.12521 

29.287 

Athletic  Order  of  Merit3 

996 

50.04568 

29.11667 

Core  GPA 

2.851132 

0.4518131 

Cumulative  GPA 

8297 

2.878256 

0.4490465 

PE  GPA 

8297 

3.119534 

0.4076689 

#  of  times  on  Superintendent's  List 

8297 

1.456912 

2.096481 

Notes:  a  The  orders  of  merit  are  normalized  to  be  percentage  rankings  in  the  class. 


The  first  group  of  covariates  consists  of  data  elements  used  by  USAFA  to 
determine  admission.  Two  elements  fall  into  this  category,  the  academic  composite  score 
and  the  weighted  composite  score.  The  academic  composite  score  is  a  numerical 
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measure  of  the  academic  qualifications  of  each  applicant.  It  is  designed  to  allow  for 
comparison  among  candidates  who  are  from  different  high  school  environments.  It  is  a 
combination  of  the  high  school  record  and  SAT  or  ACT  scores,  with  some  weighting 

factors  that  reflect  the  Academy’s  desired  areas  of  emphasis.  The  academic  composite 

2 

has  a  range  from  0  to  4000.  The  minimum  academic  composite  for  admission  is  2700 
with  an  average  score  of  3 1 90. 

The  weighted  composite  score  is  similar  to  the  academic  composite  score.  It  is  a 
numerical  measure  of  the  overall  qualifications  of  the  applicant.  The  weighted  composite 
is  a  combination  of  the  academic  composite  and  the  extracurricular  composite.3  The 
weighted  composite  is  designed  so  that  a  higher  score  corresponds  to  higher  potential  in 
an  applicant  and  is  used  in  the  decision  to  admit  specific  members  of  a  class.  While  not 
the  sole  factor  in  the  admission  decision,  it  is  a  significant  factor  and  can  be  viewed  as 
representing  the  Academy’s  assessment  of  the  applicant’s  ability  to  succeed  as  a  cadet. 
The  range  on  the  weighted  composite  is  0  to  800  with  540  being  the  minimum4  for 
admission  and  620  being  the  average  score  of  admitted  cadets. 


2  Although  the  minimum  academic  composite  is  listed  as  2700,  there  are  people  admitted 
to  the  Academy  with  scores  below  this  level.  The  2700  benchmark  is  not  a  rigid  one,  and 
can  be  waived  for  candidates  with  other  desirable  characteristics.  In  the  primary  pool 
used  in  this  study,  174  out  of  8,297  observations  (2.1%)  are  below  this  level. 

3  The  extracurricular  composite  is  a  measure  of  athletic  and  non-athletic  activities, 
including  things  such  as  varsity  letters,  club  positions,  and  other  activities.  This 
composite  is  not  used  by  itself  in  this  study. 

4  Similar  to  the  minimum  requirement  on  the  academic  composite,  this  requirement  is  not 
rigid  and  some  of  the  people  admitted  to  the  Academy  have  a  score  below  this 
requirement.  In  the  primary  pool  used  in  this  study,  213  out  of  8,297  observations  (2.6%) 
are  below  this  level. 


12 


The  next  group  is  the  group  of  covariates  that  measure  the  performance  of  the 
individual  while  at  the  Academy.  These  covariates  include  the  graduation,  academic, 
military,  and  athletic  orders  of  merit,  the  core  GPA,  the  cumulative  GPA,  the  physical 
education  GPA,  the  military  performance  average,  and  the  number  of  times  on  the 
Superintendent’s  list. 

The  various  orders  of  merit  represent  that  individual’s  standing  in  his  or  her  class 
in  that  area  at  the  time  of  graduation.  The  military  order  of  merit  is  based  on  a  rank 
ordering  of  the  military  performance  average;  likewise,  the  academic  order  of  merit  is 
based  on  a  rank  ordering  of  the  cumulative  GPA.  The  athletic  order  of  merit5  comes 
from  a  weighting  of  several  components,  specifically  the  physical  education  GPA  and 
performance  on  physical  fitness  tests.6  The  graduation  order  of  merit  is  a  weighted  score 
of  performance  in  all  three  areas,  academics,  athletics,  and  military.  The  graduation 
order  of  merit  is  used  to  determine  who  graduates  as  a  distinguished  graduate  and  also  as 
an  ordering  for  selecting  AFSCs  upon  graduation. 

Cumulative  GPA  is  the  grade  performance  in  all  academic  classes.  The  core  GPA 
is  slightly  different.  There  are  a  large  number  of  core  classes  that  all  cadets  are  required 
to  take,  regardless  of  major.  Currently,  the  core  consists  of  109  hours  of  course  work, 
including  49  semester  hours  of  engineering  and  basic  sciences,  45  hours  of  humanities 

5  A  specific  athletic  order  of  merit  was  not  assigned  to  cadets  until  1997.  Prior  to  this, 
there  was  no  specific  ordering  of  athletic  performance,  but  athletic  performance  was 
included  in  the  graduation  order  of  merit. 

6  The  fitness  tests  include  a  test  of  aerobic  fitness  and  a  test  of  strength  and  endurance. 
Each  of  the  tests  is  scored  on  a  numerical  scale  of  0  to  500,  based  on  the  performance  in 
each  event,  and  then  converted  to  a  GPA-like  scale  to  be  combined  with  the  physical 
education  grades. 
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and  social  sciences,  7.5  hours  of  Military  Strategic  Studies,  6  hours  of  physical  education 
classes,  and  1.5  hours  of  Air  Force  Operations  (US  Air  Force  Academy  Academics  Fact 
Sheet,  2001).  The  core  GPA  represents  the  academic  performance  in  these  classes  and 
can  be  used  to  compare  academic  performance  in  the  classes  that  all  cadets  have  in 
common.  The  physical  education  GPA  is  a  measure  of  performance  in  the  required 
physical  education  classes.  Cadets  must  take  three  0.5  credit  physical  education  classes  a 
year  and  receive  a  grade  based  on  performance  in  the  class.  All  three  of  these  measures 
are  on  a  standard  4.0  scale. 

The  military  performance  average  (MPA)  is  a  measure  of  the  individual’s 
performance  of  the  required  military  duties.  Cadets  hold  a  variety  of  positions  in  the 
cadet  wing,  with  the  level  of  responsibility  correlated  with  the  class  year.  The  senior 
cadets  hold  the  highest  leadership  positions,  with  junior  and  sophomore  cadets  serving  as 
staff  under  the  seniors’  leadership.  The  freshmen  cadets  do  not  hold  a  specific  position, 
but  rather  focus  on  learning  the  basics  of  military  service.  In  addition  to  specific 
requirements  based  on  the  position  a  cadet  holds,  there  are  duties  that  all  cadets  have, 
such  as  the  proper  wear  and  maintenance  of  the  uniform,  maintenance  of  the  dormitories, 
marching  and  drill  requirements,  and  knowledge  of  the  military.  The  MPA  is  the 
assessment  of  the  cadet’s  performance  in  all  of  these  areas.  The  assigning  of  an  MPA  is 
done  based  on  the  recommendations  and  evaluations  of  the  cadet’s  superiors,  including 
officers,  instructors,  and  other  cadets  in  superior  positions.  The  MPA  system  also 
operates  on  a  quota  system,  with  limited  number  of  various  scores  available  to  be 
assigned.  MPAs  are  assigned  every  semester,  as  well  as  in  the  summer  for  positions  held 
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in  military  programs.  These  individual  MPAs  are  averaged  to  yield  the  MPA  score  of  a 
cadet.  MPAs  are  assigned  on  a  4.0  scale,  similar  to  GPA. 

The  final  measure  of  performance  used  in  this  study  is  the  number  of  times  a 
cadet  appeared  on  the  Superintendent’s  list.  The  Superintendent’s  list  is  published  each 
semester  and  recognizes  outstanding  performance  in  academics,  athletics,7  and  military 
duties.  To  appear  on  the  list,  a  cadet  must  have  a  semester  GPA  over  3.0,  a  semester 
MPA  over  3.0,  and  a  semester  physical  education  score  over  3.0.  Appearance  on  the  list 
signals  the  ability  to  do  well  in  all  areas  at  the  same  time. 

The  next  data  elements  deal  with  the  military  status  of  the  cadet’s  parents.  These 
elements  contain  data  on  both  parents  and  whether  they  served  in  the  military  or  not. 
They  tell  in  which  branch  of  the  military  the  parent  served  and  the  status  of  that  service 
(active,  reserves,  national  guard,  retired,  separated,  or  deceased).  From  this  information, 
several  dummy  variables  regarding  the  military  status  of  the  parent  are  created. 

Dummies  for  any  parent  in  the  Air  Force,  any  parent  in  a  military  branch  other  than  the 
Air  Force,  any  parent  in  any  branch  of  the  military,  and  both  parents  in  the  military  are 
created.8 

Another  pair  of  covariates  deals  with  the  medical  status  of  the  cadet.  The  medical 
status  is  identified  separately  for  entry  and  graduation.  Before  entering  USAFA  and 
again  before  graduating,  cadets  must  undergo  an  extensive  physical  examination  to 
determine  their  medical  qualifications  for  military  service.  During  the  exam,  all  areas  of 

7  The  Superintendent’s  list  did  not  include  athletic  performance  until  1997.  Prior  to  this, 
it  was  based  only  on  academic  and  military  performance. 

8  These  dummies  are  not  all  completely  independent  from  each  other.  Because  of  this, 
not  all  of  them  can  be  included  in  a  single  regression  without  getting  multicollinearity. 
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health  are  studied,  including  vision,  hearing,  height,  weight,  dental  health,  the  respiratory 
system,  the  gastrointestinal  system,  the  musculoskeletal  system,  the  heart  and  vascular 
system,  and  mental  health.  The  results  determine  in  which  category  the  individual 
belongs.  Usually,  candidates  for  admission  ruled  not  qualified  will  not  be  admitted  to 
USAFA  and  USAFA  cadets  becoming  not  qualified  during  their  time  at  the  Academy 
may  be  medically  discharged.  Waivers  for  the  medical  requirements  are  available  on  a 
case-by-case  basis.  These  waivers  may  grant  cadets  a  higher  level  of  medical  clearance, 
such  as  granting  commission-qualified  cadets  pilot  qualification  or  individuals  deemed 
not  medically  qualified  commission-qualified  status.  Each  waiver  is  a  unique  decision 
and  no  general  rule  is  available  for  determining  when  a  waiver  is  granted  and  when  it  is 
not  (USAFA  Admissions  Catalog,  2001).  From  this  information  regarding  medical 
status,  several  dummy  variables  are  created.  For  the  entering  medical  status,  the 
dummies  created  represent  pilot-qualified,  pilot-qualified  with  a  waiver,  navigator- 
qualified,  navigator-qualified  with  a  waiver,  commission-qualified,  not  commission- 
qualified,  and  unknown  medical  status.  For  the  graduation  medical  status,  the  dummies 
created  are  slightly  different.  Waivered  status  is  no  longer  relevant.  Therefore,  the 
categories  are  pilot-qualified,  navigator-qualified,  commission-qualified,  disqualified, 
and  medical  status  not  available.9  Additionally,  one  more  dummy  is  created  for  the 


9  The  observations  that  have  medical  status  at  graduation  as  not  available  are  usually 
observations  where  the  cadet  out-processes  from  the  Academy  and  thus  never  graduates. 
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classification  of  medically  qualified  for  a  specialist  field,  which  is  a  more  rigorous 
qualification  than  commission  qualified.10 

The  next  group  of  data  elements  is  used  to  create  a  set  of  dummy  variables  that 
identify  the  cadet’s  graduation  status.  The  categories  created  by  these  dummy  variables 
are  graduated  and  entered  the  Air  Force,  graduated  and  entered  any  branch  of  the 
military,  including  the  Air  Force,  involuntarily  separated  from  the  Academy,  voluntarily 
separated  from  the  Academy,  and  graduated  posthumously.  The  involuntarily  separated 
designation  is  for  those  cadets  who  leave  the  Academy  not  of  their  own  volition  for 
offenses  such  as  honor  code  violations,  poor  grades,  unacceptable  athletic  performance, 
poor  military  performance,  and  criminal  infractions,  among  other  things.  Voluntarily 
separated  means  that  the  cadet  decided  for  one  reason  or  another  to  leave  the  Academy  of 
his  or  her  own  free  will  and  was  not  forced  out  for  poor  performance.1 1 

Other  data  for  the  study  provide  information  on  the  decision  to  attend  pilot 
training  following  graduation,12  the  length  of  commitment  faced  for  pilot  training,  and 

13 

macroeconomic  data  on  a  state  level. 


10  The  specialist  qualification  is  for  AFSCs  such  as  missileer  or  Air  Traffic  Controller 
that  have  more  rigorous  qualifications  than  the  commission  qualification,  but  different 
requirements  than  the  pilot  qualification  or  the  navigator  qualification. 

11  This  means  that  voluntarily  separated  cadets  can  be  viewed  as  revealing  a  preference 
for  not  serving  in  the  military,  while  no  such  conclusion  can  be  drawn  about  those  who 
are  involuntarily  separated. 

12  This  information  is  used  to  create  a  dummy,  which  applies  only  to  the  decision 
immediately  following  graduation.  It  is  possible  to  enter  pilot  training  after  starting  in 
another  career  field.  However,  this  decision  is  not  considered  in  this  study. 

13  The  macroeconomic  data,  as  is  explained  below,  is  used  in  the  interaction  terms.  The 
source  for  this  data  is  data  collected  from  county  business  patterns. 


METHODS  OF  ANALYSIS 


The  initial  data  set  gathered  for  analysis  consists  of  21,673  observations.  These 
observations  represent  all  the  individuals  who  enrolled  at  the  Air  Force  Academy  from 
1981  to  1996,  representing  the  classes  of  1985  through  2000.  The  set  is  broken  into 
several  groups  for  initial  analysis.  The  groups  of  study  are  segmented  based  on  medical 
and  graduation  status.1  During  this  initial  analysis,  several  different  medical  and 
graduation  characteristics  were  considered  for  inclusion  in  the  final  group.  The  initial 
analysis  will  not  be  described  in  great  detail  here,  but  it  consisted  of  comparing  trends  in 
the  data  between  the  different  groups  considered. 

The  result  of  this  initial  analysis  is  the  selection  of  a  group  that  had  several 
distinct  characteristics.  First,  only  those  cadets  who  graduated  from  the  Academy  and 
entered  any  branch  of  military  service  are  included.2  This  group  represents  the  part  of  the 
overall  data  set  that  was  eventually  faced  with  the  pilot  training  decision  and  made  an 
observable  choice.  While  it  can  be  argued  that  those  cadets  who  voluntarily  left  the 
Academy  also  could  be  observed  to  make  a  choice,  these  individuals  are  not  included 
because  many  times  they  lack  sufficient  data  for  inclusion  in  the  study.  For  example,  a 


1  The  groups  studied  in  this  analysis  consist  of  groups  consisting  of  as  all  observations 
who  graduated  from  the  Academy,  all  observations  who  graduated  from  the  Academy 
and  are  pilot-qualified,  all  observations  who  graduated  or  voluntarily  separated,  and  all 
observations  who  graduated  or  voluntarily  separated  and  are  pilot-qualified. 

2  The  total  number  of  people  who  graduate  from  USAFA  and  do  not  enter  the  Air  Force, 
but  enter  one  of  the  other  branches  of  military  service  is  107  out  of  the  8,297  (1.3%). 
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cadet  who  leaves  after  one  semester  does  not  provide  sufficient  performance  data  and 
thus  could  skew  the  results.  Also,  the  voluntary  separated  designation  is  sometimes 
misleading  as  individuals  sometimes  resign  rather  than  go  through  the  disenrollment 
process.  These  individuals  are  shown  to  have  left  on  their  own,  but  that  may  be  because 
they  were  going  to  be  forced  out  and  did  not  want  to  wait  around  for  the  disenrollment 
process  to  be  completed.  In  subsequent  work,  the  biases  associated  with  making  this 
exclusion  will  be  investigated  more  fully. 

The  next  characteristic  of  the  final  group  selected  for  study  is  that  all  members  are 
medically  either  pilot-qualified  at  entry  or  pilot-qualified  with  a  waiver  at  entry.  This 
excludes  any  individuals  who  attend  the  Academy,  but  are  never  physically  qualified  to 
select  pilot  training.  Including  these  people  could  skew  the  results  because  the  observed 
decision  would  be  that  these  people  chose  not  to  attend  pilot  training;  however,  they 
might  have  chosen  to  attend,  had  they  been  qualified.  Since  they  are  not  qualified,  the 
decision  is  made  for  them. 

Once  the  data  is  segmented  to  include  only  cadets  who  were  medically  qualified 
to  attend  pilot  training  at  the  time  of  entry  into  the  Academy  and  then  graduated  and 
entered  military  service,  the  number  of  observations  decreases  to  8,297.  From  these 
observations,  a  basic  model  is  created  and  analyzed  through  the  use  of  linear  regressions.3 

The  fundamental  estimating  equation  used  in  this  analysis  is: 

Pr(Pilot)  =  a  +  fiX  +  /(Commitment)  +  trend)  +  <9(pilot  control)  +  //(interactions)  +  s 


3  Several  specifications  are  also  considered  on  a  group  that  added  those  cadets  who  ended 
up  as  pilot-qualified  at  graduation.  No  significant  differences  were  noted  in  the  results. 
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where  Pr(Pilot)  represents  the  probability  of  selecting  pilot  training  as  the  initial  AFSC 
upon  graduation.  This  equation  is  estimated  using  a  linear  probability  model,  rather  than 
a  Probit  model,  because  of  the  desire  to  compare  group  means  in  the  analysis.  Future 
work  will  determine  sensitivity  to  this  linearity  assumption.  Commitment  represents  the 
length  of  commitment  in  years  owed  to  the  Air  Force  for  pilot  training.  Including  the 
trend  variable,  which  is  created  from  the  graduation  class  year,  controls  for  time  trends. 
The  vector  X  represents  a  set  of  individual  data  based  on  the  information  explained  in  the 
data  section  above.  Included  in  this  vector  are  measures  of  race,  sex,  weighted  composite 
score,  academic  composite  score,  core  GPA,  cumulative  GPA,  physical  education  GPA, 
MPA,  number  of  times  on  the  Superintendent’s  list,  parental  military  status,  graduation 
order  of  merit,  academic  order  of  merit,  and  military  order  of  merit.4  The  pilot  control 
variable  is  introduced  to  control  for  the  number  of  pilot  slots  available  in  any  given  year. 
Included  in  the  dataset  are  class  years  that  are  after  the  Cold  War.  At  this  period  in  time, 
there  was  a  massive  reduction  in  the  size  of  the  military  and  the  Air  Force  Academy  was 
not  spared  from  the  effects  of  this  reduction.  The  primary  effect  was  that,  for  several 
years,  the  number  of  pilot  slots  available  was  much  fewer  than  other  years.  To  control 
for  the  variability  in  the  number  of  pilot  slots  available,  this  variable  is  introduced.  The 
value  of  the  variable  for  each  observation  is  the  number  of  pilot  slots  available  for  the 
year  that  corresponds  to  the  observation.5  The  interaction  term  changes  with  the 

4  Athletic  order  of  merit  is  omitted  because  it  is  missing  in  the  majority  of  the  class  years. 

5  Another  control  considered  is  a  dummy  variable  that  separated  years  that  had  limited 
pilot  slots  from  those  with  a  larger  allotment.  Usually,  under  the  larger  allotment,  all 
cadets  who  desire  a  pilot  slot  can  get  one.  This  alternate  control  does  not  cause  the 
results  to  differ  significantly  from  those  obtained  with  the  control  used. 
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regression  run,  but  in  general  represents  an  interaction  between  a  variable  of  interest  and 
the  commitment  variable.  Two  main  interaction  models  are  used.  The  first  consists  of 
only  interactions  between  two  variables.  In  this  study,  the  two-way  interactions  all  deal 
with  the  commitment  length  variable  and  another  parameter  of  interest.  The  coefficient 
on  the  interaction  term  can  be  interpreted  as  the  differential  effect  of  the  policy  change 
between  the  various  levels  or  groups  of  the  parameter  of  interest  being  interacted  with  the 
commitment  length  variable.  The  second  type  of  interaction  term  consists  of  three-way 
interactions  between  variables.  In  the  three-way  model,  three  separate  variables  are 
chosen  for  interaction.  One  of  these  variables  is  always  the  commitment  length  variable. 
The  three-way  model  includes  three  different  two-way  interactions  representing  every 
possible  combination  of  the  three  variables.  Additionally,  there  is  an  interaction  term 
between  all  three  of  the  variables. 

The  primary  parameters  of  interest  in  this  study  are  the  coefficient  on 
commitment,  y,  which  represents  the  general  effect  of  the  change  in  commitment  on  the 
probability  of  selecting  pilot  training,  and  the  coefficient  on  the  interaction  terms,  p, 
which  represents  the  differing  effects  of  the  commitment  change  on  different  individuals 
who  have  a  certain  characteristic  level  of  the  variable  interacted.  By  combining  these 
coefficients,  the  effect  of  the  policy  change  can  be  evaluated. 

The  model  can  be  used  to  identify  the  effects  of  the  policy  regarding  commitment 
length  through  the  use  of  linear  regressions  on  the  dataset.  The  coefficients  from  these 
regressions  are  interpreted  to  determine  the  effects  of  a  change  in  policy.  Once  the 
coefficients  are  estimated,  the  variable  that  was  selected  for  the  interaction  term  is 
partitioned  into  levels  of  interest.  For  variables  of  a  numerical  nature,  such  as  weighted 
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composite,  the  levels  of  interest  are  the  minimum  score,  the  25th  percentile  score,  the 
median  score,  the  75th  percentile  score,  and  the  maximum  score.  For  categorical 
variables,  such  as  gender,  the  levels  of  interest  correspond  to  the  combinations  of  dummy 
variables  necessary  to  allow  the  comparisons  of  all  the  different  groups  possible.  The 
parameters  of  interest  are  then  combined  with  these  various  levels  to  determine  the  effect 
of  the  policy  on  each  level  of  individual.6  This  general  analysis  is  conducted  with  several 
different  variables  used  in  the  interaction  term  and  then  the  results  are  interpreted  to 
determine  the  effects  of  the  policy  change. 


6  For  example,  in  the  regression  interacting  weighted  composite  with  commitment  length, 
multiplying  the  value  of  the  maximum  weighted  composite  by  the  coefficient  on  the 
interaction  term  and  then  adding  the  coefficient  of  the  commitment  variable  determines 
the  effect  of  a  one  year  change  in  commitment  on  the  individual  with  the  highest 
weighted  composite.  In  equation  form: 

Effect  on  max  wgt.  comp=(max  wgt.  comp  score)* interaction  +  commitment  coefficient. 


RESULTS 


The  first  set  of  results  deals  with  a  baseline  model  without  interaction  terms.  As 
mentioned  above,  a  linear  probability  model  is  used  rather  than  a  Probit  model.  This 
decision  is  based  on  the  need  to  compare  means  in  the  study.  A  linear  probability  model 
allows  for  mean  comparisons  much  easier  than  does  a  Probit  model.  However,  in  order 
to  establish  a  comparison  between  the  two,  a  Probit  model  is  run.  The  model  run  is  a 
simple  one  that  involves  no  interaction  terms.  In  addition,  a  simple  linear  probability 
model  without  interaction  terms  is  built.  This  allows  for  two  things  to  be  accomplished. 
First,  it  allows  for  the  comparison  to  the  Probit  Model  to  demonstrate  that  there  are  not 
drastic  differences.  Second,  it  allows  for  a  basic  model  to  be  studied  without  the 
interaction  effects  to  try  to  determine  the  basic  effect  of  the  policy  change.  Table  2 
presents  the  results  of  these  two  models. 

Studying  Table  2,  it  is  seen  that  there  are  not  strong  differences  between  the 
Probit  model  and  the  linear  probability  model.  As  such,  the  linear  probability  model  is 
acceptable  to  use  and  will  be  used  for  the  reasons  stated  above.  Secondly,  the  table 
shows  that  there  is  an  overall  negative  effect  due  to  increasing  the  length  of  commitment 
owed  for  pilot  training.  The  interpretation  of  this  effect  says  that  a  one-year  increase  in 
the  length  of  commitment  owed  for  pilot  training  leads  to  approximately  a  7.7%  decrease 
in  the  likelihood  of  attending  pilot  training. 
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Table  2.  Comparison  of  Linear  Probability  and  Probit  Models. 


Variable 

Linear  Probability 
Model  Coefficient 

Probit  Model 
Coefficient 

Trend 

0.0274766 

0.0275931 

Commitment  Length 

-0.0769777 

-0.0793533 

Gender  Dummy 

0.2221058 

0.2524832 

American  Indian  Dummy 

0.0632051 

0.0623611 

Asian  Dummy 

-0.0370822 

-0.0453034 

Black  Dummy 

-0.1136575 

-0.1286032 

Hispanic  Dummy 

-0.056925 

-0.0638528 

Weighted  Composite 

0.0002901 

0.0002981 

Academic  Composite 

-0.0000498 

-0.0000518 

Core  GPA 

0.1359741 

0.1450008 

Cumulative  GPA 

-0.2779732 

-0.3154486 

PE  GPA 

0.0460006 

0.0507736 

MPA 

0.0761735 

0.1042734 

#  of  times  on  Superintendent's  List 

-0.0114722 

-0.0114476 

Parent  in  the  Air  Force  Dummy 

-0.0006294 

0.0034722 

Parent  in  Any  Military  Branch  Dummy 

0.0142074 

0.0127461 

Both  Parents  in  the  Military  Dummy 

-0.1345751 

-0.1389518 

Graduation  Order  of  Merit 

0.0032197 

0.0033004 

Academic  Order  of  Merit 

0.000468 

0.0007829 

Military  Order  of  Merit 

0.0004787 

0.0003653 

Pilot  Slots  Constraint 

0.0013612 

0.0013319 

During  the  initial  study  of  the  data,  several  variables  are  identified  as  being  the 
factors  of  interest  for  investigating  the  link  between  pilot  training  and  the  length  of 
commitment.  These  variables  include  weighted  composite  score,  academic  composite 
score,  race,1  gender,  parental  military  service,2  and  graduation  order  of  merit. 


1  While  race  was  identified  as  being  worthy  of  study,  some  simplifications  were  made. 
Due  to  the  small  proportions  of  all  of  the  minority  groups  identified,  observations  were 
lumped  into  two  categories,  namely  white  and  non-white.  Of  the  limited  dataset  of  8,297 
observations,  7,427  (89.5%)  were  identified  as  white,  314  (3.8%)  were  identified  as 
black,  304  (3.7%)  were  identified  as  Hispanic,  206  (2.5%)  were  identified  as 
Asian/Pacific  Islander,  and  46  (0.6%)  were  identified  as  Native  American. 

2  Parental  military  status  is  simplified  to  consider  only  several  categories,  including 
parent  in  the  Air  Force,  parent  in  another  military  branch,  and  parent  in  any  military 
branch. 
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Additionally,  the  effects  caused  by  macroeconomic  variations  are  deemed  worthy  of 
study.  These  variables  represent  the  terms  that  are  considered  in  the  interactions  of  the 
model  described  above. 

Weighted  composite  score,  academic  composite  score,  and  graduation  order  of 
merit  are  included  because  they  are  potential  measures  of  quality.  Weighted  composite 
and  academic  composite  are  the  Academy’s  measures  of  quality  at  entry  and  predictors  of 
success  at  the  Academy.  Weighted  composite  represents  the  Academy’s  assessment  of 
overall  quality,  while  academic  composite  represents  the  Academy’s  assessment  of 
intellectual  prowess  and  performance.  The  higher  the  score,  the  more  likely  that  person 
will  succeed  in  the  eyes  of  the  Academy.3  Graduation  order  of  merit  represents  the 
realization  of  performance  while  at  the  Academy.  Since  it  includes  an  evaluation  of  the 
three  primary  areas  of  performance  as  a  cadet,  it  is  a  measure  of  overall  performance  as  a 
cadet  and  thus  can  be  seen  as  a  measure  of  overall  aptitude  at  the  Academy.  Because  the 
goal  of  the  Academy  is  to  train  the  best  possible  officers,  this  means  that  the  order  of 
merit  can  be  seen  as  a  measure  of  officer  ability  and  potential.  Race  and  gender  are 
included  to  see  if  there  could  be  a  shift  in  the  basic  demographics  of  the  pilot  pool. 
Parental  military  status  is  included  to  determine  if  there  is  some  sort  of  information 
effect.  It  is  hypothesized  that  individuals  with  parents  in  the  military  and  in  the  Air  Force 
in  particular  are  better  informed  about  issues  such  as  the  quality  of  life  in  the  military  and 


3  As  described  above,  the  composite  scores  are  intended  to  be  predictors  of  success  while 
at  the  Academy.  A  simple  regression  analysis  shows  that,  on  the  dataset  used  in  this 
study,  weighted  composite  explains  20.62%  of  the  variation  in  graduation  order  of  merit, 
while  academic  composite  explains  24.19%  of  the  variation  in  graduation  order  of  merit. 
Additionally,  the  academic  composite  explains  26.39%  of  the  variation  in  academic  order 
of  merit.  As  such,  these  scores  are  only  moderately  accurate  in  their  stated  purpose. 
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use  this  additional  information  in  making  their  decision.  State  macroeconomic  variables 
are  included  to  determine  if  there  is  an  effect  due  to  differences  in  local  economic 
conditions. 

The  first  set  of  model  specifications  deals  with  the  performance  measures 
discussed  above.  In  these  first  regressions,  only  two-way  interaction  terms  are  included 
in  the  model.  Table  3  presents  the  results  of  these  regressions.  Specification  1  interacts 
weighted  composite  with  the  commitment  length.  In  this  regression,  the  coefficient  of 
the  length  of  commitment  is  positive  and  significant.4  This  would  seem  to  suggest  that 
increasing  the  length  of  commitment  for  pilot  training  by  one  year  would  increase  the 
likelihood  of  choosing  pilot  training  by  33.75%.  However,  this  interpretation  does  not 
account  for  the  full  effects  in  the  model  specification.  The  derivative  that  measures  the 
full  effect  is  the  linear  combination  of  all  of  the  coefficients  based  on  the  commitment 
length  variable.  This  means  the  interaction  term  must  be  accounted  for  as  well  as  the 
coefficient  on  the  commitment  length  variable.  Incorporating  the  interaction  term,  which 
is  negative  and  significant,  alters  the  interpretation  of  the  effect  of  a  change  in  the  policy 
regarding  the  length  of  commitment  for  pilot  training.  Considering  the  effects  on  the 
various  levels  of  weighted  composite  shows  the  true  effect  of  a  policy  change.  The  effect 
on  the  minimum  level  of  weighted  composite  observed  is  positive,  but  insignificant.5  At 
all  of  the  other  levels  considered  in  the  table,  the  effect  is  negative  and  significant. 


4  Significance  is  based  on  a  10%  level  of  significance.  This  level  will  be  applied 
throughout  the  following  discussion.  All  P-values  discussed  are  based  on  two-tailed 
tests. 

5  The  effect  on  the  Academy’s  declared  minimum  weighted  composite  score  of  540  is 
also  insignificant  (P-Value=0.167),  but  negative  (-2.55%). 
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Looking  at  the  entire  dataset,  only  1 1  (0.13%)  of  the  observations  have  weighted 
composites  that  give  an  overall  positive  effect  and  8,014  (96.59%)  of  the  observations 
have  weighted  composite  scores  that  lead  to  a  negative  and  significant  effect  due  to  a 
policy  change. 


Table  3.  Performance  Specifications. 


Dependent  Varia 

ble:  Pr  (Pilot  1 

"raining) 

Specification 

1 

2 

3 

0.010 

0.094 

0.089 

WBM 

-0.0006723 

0.002 

Commitment  x 

Ac  Composite 

-0.0000799 

0.094 

Commitment  x 

Order  of  Merit 
(percentile) 

-0.0009791 

0.001 

Minimum 

ms 

25th  Percentile 
of  Variable 

-0.0572663 

-0.0615382 

-0.0562328 

0.000 

0.000 

0.000 

50th  Percentile 
of  Variable 

-0.0754853 

-0.0758329 

-0.0801737 

0.000 

0.000 

0.000 

75th  Percentile 
of  Variable 

-0.0937715 

-0.0904472 

-0.1037283 

0.000 

0.000 

0.000 

Maximum 

-0.1692693 

-0.1426750 

-0.1283102 

0.000 

0.000 

0.000 

Note:  Each  Column  represents  a  separate  regression.  Coefficients  are  listed  on  top  with 
P-values  below. 


This  means  the  policy  change  causes  a  negative  impact  on  the  decision  to  attend 
pilot  training  for  nearly  all  cadets.  The  effect  varies  with  the  level  of  the  weighted 
composite  score  and  this  variation  is  significant,  as  shown  by  the  significance  of  the 
interaction  term  in  Specification  1.  The  effect  becomes  more  negative  with  higher 
weighted  composite  scores.  In  summary,  increasing  the  length  of  the  commitment  owed 
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for  pilot  training  will  cause  an  individual  to  be  less  likely  to  elect  to  go  to  pilot  training, 
and  this  effect  becomes  more  pronounced  as  the  cadet’s  ability,  as  measured  by  weighted 
composite,  improves. 

Specification  2  interacts  the  academic  composite  score  with  the  commitment 
length  variable.  Like  Specification  1,  the  coefficient  on  the  length  of  commitment  is 
positive  and  significant,  but  the  interaction  coefficient  balances  out  this  positive  effect. 
The  effect  on  the  minimum  academic  composite  score  in  the  dataset  is  positive,  but 
insignificant.6  At  all  of  the  other  levels  presented  in  the  table,  the  effect  of  a  one-year 
change  in  policy  is  negative  and  significant.  In  the  entire  dataset,  2  (0.02%)  of  the 
observations  have  academic  composite  scores  that  show  a  positive  effect  and  8,208 
(98.93%)  of  the  observations  have  academic  composite  scores  that  lead  to  a  negative  and 
significant  effect  due  to  the  policy  change. 

Like  Specification  1,  the  results  of  Specification  2  show  a  negative  impact  due  to 
the  increased  length  of  commitment.  The  effect  varies  with  the  academic  composite 
score,  becoming  more  negative  with  higher  academic  composite  scores.  The  variation  in 
effect  due  to  the  level  of  the  academic  composite  score  is  significant,  as  shown  by  the 
significance  of  the  interaction  term. 

The  graduation  order  of  merit  interacts  with  the  commitment  length  in 
Specification  3.  In  this  regression,  the  coefficient  of  the  commitment  length  variable  is 
negative  and  significant.  Additionally,  the  coefficient  on  the  interaction  term  is  negative 
and  significant.  This  leads  to  an  overall  negative  impact  of  the  policy  change  on  the 


6  The  effect  on  the  Academy’s  declared  minimum  academic  composite  score  of  2700  is 
negative  (-3.81%)  and  significant  (P-value=0.040). 
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decision  to  attend  pilot  training.  This  impact  is  observed  at  all  the  possible  levels  of 
graduation  order  of  merit.  Specification  3  shows  similar  results  to  Specifications  1  and  2. 
There  is  a  negative  impact  associated  with  increasing  the  length  of  commitment  owed  for 
pilot  training.  This  effect  changes  with  the  order  of  merit  at  graduation.  Those 
individuals  who  graduate  higher  in  the  class  respond  more  than  their  lower-ranked 
classmates.  Since  the  coefficient  on  the  interaction  term  is  significant,  the  variation  in 
impact  is  significant. 

An  interesting  question  regards  the  exact  size  of  the  magnitude  of  the  effects 
observed.  Because  69.54%  of  those  cadets  in  the  top  third  of  weighted  composite  score 
attend  pilot  training,  this  can  be  viewed  as  the  probability  of  an  individual  in  the  top  third 
attending  pilot  training.  The  observed  effect  of  shift  in  policy  on  the  individual  with  the 
75th  percentile  value  of  weighted  composite  is  -9.38%.  This  means  that  the  policy 
change  causes  a  13.48%  decrease  in  the  likelihood  of  attending  pilot  training.  For  the 
median  individual,  the  policy  change  reduces  the  likelihood  of  going  to  pilot  training  by 
7.55%.  The  overall  likelihood  of  attending  pilot  training  for  the  middle  third  is  71.71%. 
This  means  that  the  policy  change  causes  a  10.53%  decrease  in  the  likelihood  of 
attending  pilot  training.  The  policy  reduces  the  likelihood  of  the  individual  with  the  25th 
percentile  score  attending  pilot  training  by  5.73%,  while  the  overall  likelihood  of 
attending  pilot  training  for  an  individual  in  the  bottom  third  of  weighted  composite  is 
72.86%.  This  means  that  the  policy  change  causes  a  7.86%  decrease  in  the  likelihood  of 
attending  pilot  training. 

Likewise,  the  magnitude  of  the  impact  must  be  considered  on  the  specification 
regarding  academic  composite  score.  Because  71.06%  of  those  cadets  in  the  top  third  of 
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academic  composite  score  attend  pilot  training,  this  can  be  viewed  as  the  probability  of 
an  individual  in  the  top  third  attending  pilot  training.  The  observed  effect  of  shift  in 
policy  on  the  individual  with  the  75th  percentile  value  of  academic  composite  is  -9.04%. 
This  means  that  the  policy  change  causes  a  12.73%  decrease  in  the  likelihood  of 
attending  pilot  training.  For  the  median  individual,  the  policy  change  reduces  the 
likelihood  of  going  to  pilot  training  by  7.58%.  The  overall  likelihood  of  attending  pilot 
training  for  the  middle  third  is  71.89%.  This  means  that  the  policy  change  causes  a 
10.55%  decrease  in  the  likelihood  of  attending  pilot  training.  The  policy  reduces  the 
likelihood  of  the  individual  with  the  25th  percentile  score  attending  pilot  training  by 
6.15%,  while  the  overall  likelihood  of  attending  pilot  training  for  an  individual  in  the 
bottom  third  of  academic  composite  is  71 . 1 1%.  This  means  that  the  policy  change  causes 
an  8.65%  decrease  in  the  likelihood  of  attending  pilot  training. 

Similarly,  the  magnitude  of  the  impact  must  be  considered  on  the  specification 
regarding  graduation  order  of  merit.  Because  76.81%  of  those  cadets  in  the  top  third  of 
graduation  order  of  merit  attend  pilot  training,  this  can  be  viewed  as  the  probability  of  an 
individual  in  the  top  third  attending  pilot  training.  The  observed  effect  of  shift  in  policy 
on  the  individual  with  the  75th  percentile  value  of  order  of  merit  is  -10.37%.  This  means 
that  the  policy  change  causes  a  13.50%  decrease  in  the  likelihood  of  attending  pilot 
training.  For  the  median  individual,  the  policy  change  reduces  the  likelihood  of  going  to 
pilot  training  by  8.02%.  The  overall  likelihood  of  attending  pilot  training  for  the  middle 
third  is  72.46%.  This  means  that  the  policy  change  causes  an  1 1.06%  decrease  in  the 
likelihood  of  attending  pilot  training.  The  policy  reduces  the  likelihood  of  the  individual 
with  the  25th  percentile  score  attending  pilot  training  by  5.62%,  while  the  overall 
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likelihood  of  attending  pilot  training  for  an  individual  in  the  bottom  third  of  order  of 
merit  is  64.62%.  This  means  that  the  policy  change  causes  an  8.70%  decrease  in  the 
likelihood  of  attending  pilot  training. 

The  negative  impact  of  the  policy  change  observed  in  Specifications  1, 2,  and  3 
seems  logical.  The  increased  commitment  length  increases  the  opportunity  cost  of 
attending  pilot  training.  As  pilot  training  becomes  more  “expensive”  to  attend,  the 
likelihood  of  attending  will  decrease.  What  is  more  interesting  in  these  specifications  is 
the  interaction  effect  observed  in  each  regression.  Not  only  is  there  a  negative  effect  of 
increasing  the  commitment  length,  the  effect  is  stronger  on  the  individuals  with  higher 
weighted  composite  scores,  academic  composite  scores,  and  graduation  orders  of  merit. 
Since  these  scores  can  be  viewed  as  measures  of  quality  of  the  individual,  the  implication 
is  that  the  policy  change  is  decreasing  the  quality  of  the  pool  of  individuals  who  select  to 
attend  pilot  training. 

Table  4  presents  the  results  from  the  next  group  of  regressions.  These 
specifications  deal  with  the  demographics  of  the  pilot  pool  and  the  informational  effects 
due  to  a  parent  serving  in  the  military.  Specification  4  looks  at  the  interaction  between 
the  racial  category  and  the  commitment  length.  The  coefficients  on  the  commitment 
length  and  the  interaction  variable  are  negative,  but  not  significant.  This  leads  to  the 
effect  on  nonwhites  being  negative,  but  not  significant,  while  the  effect  on  whites  is 
negative  and  significant.  Since  the  interaction  coefficient  is  not  significant,  the  different 
impacts  on  the  two  groups  are  not  statistically  significantly  different  from  each  other. 

Specification  5  interacts  gender  with  the  commitment  length.  In  this  regression, 
the  coefficient  on  the  commitment  length  is  negative  and  significant,  while  the  interaction 
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term  is  negative  but  insignificant.  The  effect  on  males  is  negative  and  significant,  as  is 
the  effect  on  females.  However,  the  two  effects  are  not  significantly  different  from  each 
other,  since  the  coefficient  on  the  interaction  variable  is  insignificant. 


Table  4.  Demographic  and  Informational  Specifications. 


Dependent  Variable:  Pr  (Pilot  Training) 

Specification 

4a 

5b 

6C 

T 

8C 

9d 

Commitment 

-0.0369090 

-0.0935967 

-0.0812594 

-0.07676 

-0.0838194 

-0.0839156 

0.174 

0.003 

0.000 

0.000 

0.000 

0.000 

Commitment  x 
Race 

-0.0447087 

0.114 

Commitment  x 
Gender 

0.01802 

0.579 

Commitment  x 
Parent  in  the  Air 
Force 

0.0247067 

0.0178063 

0.281 

0.566 

Commitment  x 

Parent  in  the 
Military  (Non  AF) 

1818ISBH33 

■m 

Commitment  x 
Parent  in  Any 
Branch 

0.0214498 

0.0097938 

0.253 

0.704 

-0.0369090 

-0.0935967 

-0.0812594 

0.174 

0.003 

0.000 

0.000 

-0.0816177 

-0.0755767 

■gjgjg|| 

-0.0851330 

0.000 

0.000 

Group  3 

-0.0563156 

0.008 

Note:  Each  column  represents  a  separate  regression.  Coefficients  are  listed  on  top  with 
P-values  listed  below. 

a.  Group  1  is  white.  Group  2  is  non-white 

b.  Group  1  is  male.  Group  2  is  female. 

c.  Group  1  is  no  military  affiliation.  Group  2  is  specified  military  affiliation. 

d.  Group  1  is  no  military  affiliation.  Group  2  is  Air  Force  only.  Group  3  is  any  military 
affiliation. 


Specifications  4  and  5  both  show  some  negative  effect  due  to  increasing  the 
commitment  length  for  pilot  training.  In  each  case,  there  is  not  a  statistically  significant 
difference  between  the  groups.  It  appears  that  the  Air  Force’s  decision  to  alter  the  length 
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of  commitment  owed  for  pilot  training  does  not  impact  any  specific  racial  group  more 
than  another.  Nor  does  the  policy  change  impact  one  gender  more  than  the  other.  This 
means  that  the  policy  change  does  not  appear  to  significantly  alter  the  demographic 
makeup  of  the  pilot  pool. 

Specifications  6,  7,  8,  and  9  study  parental  military  status.  Specification  6 
interacts  the  dummy  for  a  parent  in  the  Air  Force  with  the  commitment  length  variable. 

In  this  regression,  the  coefficient  on  the  commitment  variable  is  negative  and  significant. 
The  interaction  variable  coefficient  limits  the  effect,  as  it  is  positive,  but  it  is  statistically 
insignificant.  The  dummy  for  a  parent  in  a  branch  of  the  military  other  than  the  Air  Force 
interacts  with  the  commitment  length  variable  in  Specification  7 .  The  coefficient  on  the 
commitment  length  is  negative  and  significant  while  the  coefficient  on  the  interaction 
variable  is  negative  and  insignificant.  In  Specification  8,  the  commitment  length  is 
interacted  with  the  dummy  for  a  parent  in  any  branch  of  the  military.  In  this  case,  the 
coefficient  on  the  commitment  variable  is  negative  and  significant,  but  the  interaction 
coefficient  is  positive  and  insignificant. 

Specifications  6,  7,  and  8  all  show  the  negative  impact  of  increasing  the 
commitment  required  for  pilot  training.  Since  the  coefficients  on  the  interaction  terms 
are  always  insignificant,  there  is  not  a  statistically  significant  difference  due  to  having  a 
parent  with  military  service. 

The  variation  in  signs  on  the  interaction  terms  in  Specifications  6  and  7  motivate  a 
further  regression,  which  is  Specification  9.  In  this  regression,  dummies  for  both  a  parent 
in  the  Air  Force  and  a  parent  in  any  military  branch  are  included.  Each  of  these  dummies 
interacts  with  the  commitment  variable.  In  this  regression,  the  coefficient  on  the 
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commitment  variable  is  once  again  negative  and  significant.  The  coefficients  on  the  two 
interaction  terms  are  positive  but  insignificant.  This  regression  allows  study  of  three 
categories  of  people,  namely  those  without  a  parent  in  the  military,  those  with  a  parent  in 
the  Air  Force,  and  those  with  a  parent  in  any  branch  of  the  military,  including  the  Air 
Force.  Since  the  coefficients  on  the  interaction  terms  are  insignificant,  there  is  not  a 
statistically  different  effect  of  the  policy  change  on  the  members  of  these  three  groups. 

Like  Specifications  4  and  5,  Specifications  6,  7,  8,  and  9  show  that  the  policy 
change  has  a  negative  effect  on  the  decision  to  attend  pilot  training,  but  there  is  no 
statistically  significant  difference  in  the  possible  groupings.  This  means  that  there  is  no 
strong  difference  in  the  pilot  training  decision  given  an  informational  advantage  of 
familiarity  with  military  service. 

Table  5  presents  the  results  for  several  three-way  interaction  regressions. 
Specification  10  focuses  on  the  effects  of  the  weighted  composite  score  and  graduation 
order  of  merit  simultaneously.  This  regression  is  conducted  to  allow  a  study  of  the 
different  effects  on  the  two  general  measures  of  quality.  There  is  some  correlation 
between  graduation  order  of  merit  and  the  weighted  composite  score,  but  it  is  not  perfect 
correlation.  This  interaction  regression  allows  for  a  study  of  different  relevant  positions 
for  both  measures.  In  this  regression,  the  coefficients  are  mostly  insignificant.  The  only 
significant  coefficient  is  the  one  of  the  three-way  interaction  variable. 

The  more  interesting  results  of  this  specification  come  from  looking  at  the  effects 
on  the  various  levels  of  the  two  measures.  The  weighted  composite  score  is  the 
Academy’s  predictor  of  an  individual’s  chances  of  success  while  at  the  Academy.  The 
order  of  merit  represents  the  Academy’s  assessment  of  the  realization  of  success  while  at 
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the  Academy.  Combining  these  two  measures  allows  for  the  study  of  four  types  of 
individuals.  These  four  types  are  the  individual  who  comes  in  highly  touted  and  delivers 
with  a  high  level  of  performance,  the  individual  who  comes  in  highly  touted,  but  fails  to 
perform  up  to  expectations,  the  individual  who  is  not  predicted  to  do  as  well,  but  exceeds 
expectations  and  performs  well,  and  the  individual  who  is  not  predicted  to  do  as  well  and 
performs  to  these  expectations.7  The  strongest  negative  impact  of  a  policy  change  is 
observed  on  the  first  individual,  namely  the  one  with  a  high  weighted  composite  score 
and  a  high  graduation  order  of  merit.  The  next  strongest  impact  is  observed  on  the  third 
individual,  which  is  the  one  who  comes  in  with  a  lower  weighted  composite  score,  but 
graduates  with  a  high  order  of  merit.  Following  this  individual,  the  individual  who  enters 
with  a  high  weighted  composite  score,  but  graduates  with  a  lower  order  of  merit, 
experiences  the  next  strongest  effect.  The  weakest  effect  is  observed  on  the  individual 
who  comes  in  with  a  low  weighted  composite  score  and  leaves  with  a  low  graduation 
order  of  merit. 

These  results  have  an  important  implication.  The  effect  is  stronger  on  people  who 
improve  versus  those  who  decline,  which  would  seem  to  hurt  the  Air  Force.  Someone 
who  performed  relatively  poorly  in  high  school,  earning  a  low  weighted  composite  score 
entering  the  Academy,  but  then  improved,  either  because  he  turned  around  his 
performance,  found  his  element,  or  some  other  similar  change,  and  thus  graduated  higher 


7  In  considering  the  cases  described  here,  the  strength  of  the  effects  also  depend  on  the 
numbers  in  each  group.  In  the  dataset  used  for  this  study,  425  individuals  out  of  8,297 
(5.1%)  transitioned  from  the  top  third  in  weighted  composite  to  the  bottom  third  in  order 
of  merit.  The  number  transitioning  from  the  bottom  third  in  weighted  composite  to  the 
top  third  in  order  of  merit  is  406  out  of  8,297  (4.9%).  Overall,  2,160  individuals  (26.0%) 
transition  up  at  least  a  third,  while  2,128  individuals  (25.6%)  transition  down  at  least  a 
third.  Based  on  this,  the  effects  observed  can  be  serious. 


35 


than  his  entering  performance  measurement  predicted  would  seem  to  be  more  desirable 
than  someone  who  came  in  with  a  high  performance  rating,  but  then  failed  to  back  that 
previous  performance  and  its  expectations  up  while  at  the  Academy.  This  leads  to  a 
question  of  whether  someone  on  the  rise  is  better  than  someone  who  is  declining  in 
performance.  It  would  seem  logical  that  the  person  improving  and  ending  with  a  better 
performance  would  be  more  desirable  than  the  person  who  declined  and  ended  with  the 
poorer  performance.  It  can  even  be  questioned  whether  the  declining  individual  is  the 
least  desirable  individual  of  the  four  described  above.  The  improving  performer  and  the 
consistently  high  performer  are  logically  more  desirable  than  the  declining  performer.  It 
is  possible  that  the  consistent  relatively  low  performer  is  better  as  well  as  this  individual 
performs  to  his  expected  level.  This  level  of  performance  is  sufficient  because  this 
individual  still  manages  to  successfully  complete  the  training  program  of  the  Academy. 
The  declining  performer  attained  a  sufficient  level  of  performance  to  graduate  from  the 
Academy,  but  might  continue  to  decline  to  a  level  where  his  performance  does  not  meet 
standards.  Because  of  this,  the  individual  who  is  consistent  could  arguably  be  more 
desirable  than  the  declining  performer. 

Specifications  1 1  and  12  discuss  the  three-way  interactions  between  performance 
and  the  military  status  of  the  parents.  Since  the  two-way  interactions  showed  no 
significant  variation  in  the  difference  between  the  different  branches  of  military  service, 
the  only  category  considered  here  is  a  parent  in  any  branch  of  the  military.  These 
regressions  allow  for  a  comparison  between  the  performance  effect  and  the  information 
effect  of  having  a  parent  in  the  military. 
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Table  5.  Three-Way  Regressions. 


Dependent  Variable:  Pr  (Pilot  Training 

_ 

Specification 

10 

11 

12 

Commitment 

0.0238527 

0.4083014 

-0.0292472 

0.898 

0.003 

0.137 

Commitment  x  Wgt 
Composite 

-0.0001053 

-0.0007981 

0.738 

0.000 

Commitment  x  Order 
of  Merit  (percentile) 

0.003 

-0.001166 

0.165 

0.000 

Commitment  x  Parent 
in  Any  Branch 

-0.172517 

0.0115264 

0.254 

0.650 

Commitment  x  Wgt 
Composite  x  Order  of 
Merit 

-0.0000064 

0.089 

Commitment  x  Wgt 
Composite  x  Parent  in 
Any  Branch 

0.0003189 

0.201 

Commitment  x  Order 
of  Merit  x  Parent  in 

Any  Branch 

0.0003156 

0.367 

Note:  Each  column  represents  a  separate  regression.  Coefficients  are  listed  on  top  with 
P-values  below. 


Specification  1 1  interacts  the  weighted  composite  score  with  the  dummy  for  a 
parent  in  any  branch  of  the  military.  In  this  regression,  the  coefficient  on  the 
commitment  variable  is  positive  and  significant.  For  the  two-way  interaction  variables, 
both  are  negative,  but  only  the  coefficient  of  the  variable  for  interaction  between 
commitment  length  and  weighted  composite  score  is  significant.  The  three-way 
interaction  coefficient  is  positive,  but  insignificant. 

In  this  regression,  the  strongest  effect  is  observed  on  the  individual  with  the 
highest  weighed  composite  and  without  a  parent  with  military  service.  The  smallest 
effect  is  observed  on  the  individual  with  the  lowest  weighted  composite  and  a  parent  who 
served  in  any  military  branch.  The  differential  effect  between  having  a  parent  in  the 
military  and  not  having  one  is  larger  with  higher  weighted  composite  scores.  This  would 
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suggest  that  the  informational  effect  is  larger  for  the  individuals  with  higher  weighted 
composite  scores;  however  the  coefficients  are  not  statistically  significant.  The 
regression  also  shows  the  increasing  negative  effect  on  the  better  caliber  individuals. 

The  interaction  between  graduation  order  of  merit  and  the  dummy  for  a  parent  in 
the  military  is  considered  in  Specification  12.  In  this  regression,  the  only  statistically 
significant  coefficient  is  the  coefficient  on  the  two-way  interaction  between  the 
commitment  variable  and  the  graduation  order  of  merit.  All  other  coefficients  are 
insignificant. 

The  results  of  Specification  12  are  identical  to  those  of  Specification  1 1 .  The 
strongest  effect  is  observed  on  the  individual  with  the  highest  graduation  order  of  merit 
and  without  a  parent  with  military  service.  The  smallest  effect  is  observed  on  the 
individual  with  the  lowest  order  of  merit  and  a  parent  who  served  in  any  military  branch. 
The  differential  effect  between  having  a  parent  in  the  military  and  not  having  one  is 
larger  with  higher  orders  of  merit.  This  would  suggest  that  the  informational  effect  is 
larger  for  the  individuals  with  higher  orders  of  merit;  however  the  coefficients  are  not 
statistically  significant.  The  regression  also  shows  the  increasing  negative  effect  on  the 
better  caliber  individuals. 

Specifications  13,  14,  and  15  consider  the  effects  of  the  local  macroeconomic 
conditions  on  the  decision  to  attend  pilot  training  by  looking  at  the  unemployment  rate  in 
each  individual’s  home  state.8  The  results  of  these  regressions  are  presented  in  Table  6. 

8  A  regression  was  also  run  with  the  state  employment  growth  rate  as  the  macroeconomic 
variable  used.  However,  the  results  did  not  significantly  differ  from  those  obtained  using 
the  unemployment  rate.  The  unemployment  rate  is  used  and  discussed  because  it  is  a 
more  common  macroeconomic  measure. 
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Specification  13  consists  of  a  two-way  interaction  between  the  unemployment  rate  and 
the  commitment  length  variable.  In  this  regression,  the  coefficient  on  the  commitment 
variable  is  negative  and  significant,  as  is  the  coefficient  on  the  interaction  term.  This 
specification  shows  the  negative  impact  of  increasing  the  length  of  commitment.  In 
addition,  it  shows  that  the  negative  effect  is  stronger  on  those  individuals  who  come  from 
states  with  higher  unemployment  rates.  Because  the  interaction  term  is  significant,  the 
difference  in  the  effect  due  to  different  levels  of  unemployment  is  significant  as  well. 

Specification  14  consists  of  a  three-way  interaction  between  the  unemployment 
rate,  the  weighted  composite  score,  and  the  commitment  length  variable.  In  this 
regression,  the  coefficient  on  the  commitment  variable  is  positive  and  significant.  The 
two-way  interaction  coefficients  between  commitment  length  and  unemployment  rate  and 
commitment  length  and  weighted  composite  are  both  negative  and  significant.  This 
shows  that  the  individual  effect  on  the  decision  to  attend  pilot  training  increases  with 
higher  weighted  composite  scores  and  unemployment  rates.  The  coefficient  on  the  three- 
way  interaction  variable  is  positive  and  significant.  Thus,  this  interaction  term  serves  to 
limit  the  negative  effect  on  an  individual  with  a  set  weighted  composite  score  when  the 
unemployment  rate  facing  that  individual  is  increased.  Likewise,  for  a  fixed 
unemployment  rate,  the  effect  of  the  policy  change  is  limited  when  the  weighted 
composite  is  increased.  The  overall  impact  of  a  policy  change  is  governed  by  the 
combination  of  all  of  these  factors.  In  this  specification,  the  strongest  effect  is  seen  on 
the  combination  of  the  highest  weighted  composite  score  and  the  highest  unemployment 
rate.  Reducing  the  weighted  composite  score  has  more  of  a  mitigating  effect  than 
reducing  the  unemployment  rate. 
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Table  6.  Macroeconomic  Specifications. 


Dependent  Variable:  Pr  ( 

Pilot  Training) 

Specification 

13 

14 

15 

Commitment 

-0.0497727 

0.5533899 

0.0424036 

0.009 

0.000 

0.102 

Unemployment  Rate 

0.0373483 

0.3897074 

maim 

0.103 

0.011 

Commitment  x 
Unemployment  Rate 

-0.0062606 

-0.0614345 

-0.0191732 

0.099 

0.002 

0.000 

Commitment  x  Weighted 
Composite 

-0.0009768 

0.000 

Commitment  x  Order  of 
Merit 

-0.001796 

0.000 

Wgt.  Composite  x 
Unemployment  Rate 

-0.0005751 

0.021 

Order  of  Merit  x 
Unemployment  Rate 

-0.0014106 

0.000 

Commitment  x  Wgt 
Composite  x 
Unemployment  Rate 

0.0000903 

0.005 

Commitment  x  Order  of 
Merit  x  Unemployment 
Rate 

0.0002421 

0.000 

Note:  Each  column  represents  a  separate  regression.  Coefficients  are  listed  on  top,  with 
P-values  listed  below. 


Specification  15  looks  at  the  interaction  between  unemployment  rate,  graduation 
order  of  merit,  and  commitment  length.  In  this  regression,  the  coefficient  on  commitment 
length  is  positive,  but  slightly  above  the  threshold  for  significance.  The  two-way 
interaction  coefficients  between  commitment  length  and  the  other  two  variables  are  both 
negative  and  significant.  The  coefficient  on  the  three-way  interaction  variable  is  positive 
and  significant.  This  leads  to  similar  results  to  Specification  14.  The  strongest  effect  is 
seen  on  the  individual  with  a  high  graduation  order  of  merit  facing  a  high  unemployment 
rate.  Reducing  the  order  of  merit  limits  the  effects  of  the  policy  change  more  than 
reducing  the  unemployment  rate. 
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Each  of  the  macroeconomic  specifications  yields  some  interesting  results.  The 
three-way  regressions  show  that  the  impact  is  higher  on  the  higher  ability  individuals. 

This  result  has  been  observed  in  previous  specifications.  An  interesting  result  appears 
when  studying  the  effects  on  individuals  facing  the  different  levels  of  unemployment.  In 
each  specification,  a  higher  unemployment  rate  leads  to  a  stronger  impact  on  an 
individual  making  the  pilot  training  decision.  This  seems  counter  to  logic  as  a  higher 
unemployment  rate  would  seem  to  present  fewer  job  prospects.  In  a  normal  situation,  the 
effect  would  be  higher  for  a  lower  unemployment  rate  as  increasing  the  commitment 
length  causes  an  increase  in  the  opportunity  cost.  This  increased  opportunity  cost  would 
have  a  more  pronounced  effect  in  low  unemployment  situations,  as  the  opportunity  costs 
are  higher  in  this  situation.  However,  this  is  not  the  effect  that  is  observed.  There  are 
several  possible  reasons  for  this  difference.  One  possible  explanation  is  that  the 
individuals  in  the  dataset  for  this  study  tend  to  be  the  more  elite  graduates  from  their  local 
high  schools.  As  such,  they  might  feel  that  they  have  good  job  prospects  regardless  of 
what  the  unemployment  rate  is,  as  long  as  it  remains  within  reason,  as  is  the  case  in  the 
years  observed  for  this  study.  Thus,  they  ignore  the  unemployment  rate  in  their  decision. 
Another  issue  deals  with  the  timing  of  the  unemployment  rate  versus  the  decision  timing. 
This  study  used  the  unemployment  rate  that  corresponds  to  the  rate  that  exists  during  the 
year  of  graduation.  However,  this  rate  is  unknown  at  the  time  of  entry  to  the  Academy 
and  by  the  time  it  is  known,  the  individual  has  already  incurred  some  commitment  to  the 
Air  Force  and  thus  will  not  be  seeking  a  job  in  the  civilian  world  for  some  time.  Another 
possible  explanation  is  that  the  individuals  who  are  least  able  to  respond  to  the 
unemployment  rate  in  the  future  are  impacted  by  it  more.  They  might  respond  more  now 
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because  of  the  perceived  need  to  make  more  money  now  by  getting  out  of  the  Air  Force 
sooner  because  they  are  lesser  ability  and  more  likely  to  suffer  under  high  unemployment 
rates.  In  each  of  the  three-way  specifications,  the  effect  of  changing  the  measure  of 
ability  leads  to  a  greater  change  in  effect  than  changing  the  unemployment  rate.  This 
suggests  that  an  individual’s  abilities  play  a  larger  role  than  the  local  economic 
conditions. 


ALTERNATIVE  SPECIFICATIONS 

In  addition  to  the  specifications  explained  above,  several  others  are  generated  to 
look  at  specific  issues.  As  was  explained  before,  the  dataset  is  limited  to  those  cadets 
who  graduated  and  entered  military  service  and  are  pilot-qualified  at  the  time  of  entry 
into  the  Air  Force  Academy.  The  reasons  for  this  limitation  were  explained  above. 
However,  the  group  for  analysis  is  expanded  to  include  all  the  cadets  who  graduated  and 
entered  the  military  and  were  either  pilot-qualified  at  the  time  of  entry  or  the  time  of 
graduation.  This  group  is  studied  to  insure  that  there  are  not  any  strong  deviations  from 
the  smaller  group  and  thus  check  the  validity  of  limiting  the  dataset.  The  same  procedure 
for  analysis  as  outlined  above  is  conducted  on  this  dataset.  The  results  of  the  analysis 
yield  similar  results  as  the  analysis  of  the  primary  group. 

Another  analysis  is  conducted  using  dummies  for  the  different  states  incorporated 
into  the  model.  The  regressions  from  this  analysis  allow  the  possibility  of  differential 
effects  of  the  policy  change  on  residents  of  different  states  to  be  considered.  The  results 
of  this  analysis  do  not  substantially  differ  from  the  results  of  the  analysis  of  the  original 
group.  Because  of  this,  it  can  be  concluded  that  there  is  not  a  different  effect  based  on 
the  state  from  which  a  cadet  comes. 

The  final  set  of  alternative  regressions  incorporates  the  state  macroeconomic 
variables  described  above  into  all  of  the  regressions  as  an  additional  covariate  in  the 
model.  Analysis  of  the  results  of  these  regressions  shows  no  substantial  differences  from 
the  results  of  the  initial  analysis.  Because  of  this,  the  state  macroeconomic  variables  are 
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not  directly  incorporated  into  the  model,  but  are  still  used  in  the  interaction  terms  of  some 
specifications. 

The  results  of  all  of  these  alternative  specifications  show  that  there  are  not 
substantial  differences  from  the  original  model  used  and  thus  these  issues  are  not 
necessary  to  account  for  in  the  study. 


POTENTIAL  POINTS  OF  CONTENTION 
There  are  some  points  of  potential  contention  with  this  study.  These  issues  will 
be  addressed  here.  The  first  problem  is  the  issue  of  group  selection.  This  issue  is  a  two¬ 
fold  issue.  The  issue  of  selecting  the  Air  Force  Academy,  as  was  explained  above,  can  be 
potentially  troubling.  A  better  study  of  the  issue  would  look  at  a  larger  group  including 
representatives  from  all  commissioning  sources.  The  effects  explained  in  this  thesis 
apply  only  to  the  group  studied.  There  could  be  different  effects  on  pilot  candidates  who 
come  from  other  commissioning  sources.  To  make  a  full  evaluation  of  the  policy,  these 
other  groups  should  be  considered  as  well.  However,  as  mentioned  above,  using  only 
USAFA  graduates  helps  to  ensure  comparability  among  the  study  subjects. 

Another  issue  regarding  group  selection  is  the  decision  to  limit  the  Academy  pool 
further  by  considering  only  those  cadets  who  graduate  and  enter  military  service  and  are 
pilot-qualified  at  entry.  A  better  study  would  include  all  cadets  at  the  Academy. 
However,  the  data  available  is  insufficient  for  such  a  study.  The  medical  clearance 
question  is  difficult  to  handle.  The  physical  required  for  graduation  is  less  in-depth  than 
the  one  required  for  pilot  training.  Thus,  obtaining  a  pilot-qualified  medical  clearance 
reflects  some  interest  in  becoming  a  pilot.  Some  people  who  alter  their  decision  based  on 
the  policy  change  might  not  have  been  picked  up  by  this  study  because  they  made  their 
decision  early  and  went  for  the  less  comprehensive  physical. 

This  leads  to  another  problem,  namely  the  problem  of  identifying  the  motivating 
factors  in  the  decision  to  attend  pilot  training.  The  length  of  commitment  is  only  one  part 
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that  must  be  considered  in  making  the  decision.  A  better  study  would  conduct  interviews 
of  the  people  making  the  decision  at  the  time  they  make  it  to  attempt  to  weigh  the 
importance  of  the  commitment  length,  and  thus  the  policy  change,  in  the  decision  to 
attend  pilot  training. 

Another  area  of  contention  is  the  interpretation  of  the  weighted  composite  score, 
academic  composite  score,  and  graduation  order  of  merit  as  measures  of  quality.  These 
measures  are  selected  because  they  are  used  by  the  Academy  as  measures  of  quality,  but 
they  are  not  perfect.  They  are  subject  to  extremely  complex  interactions  between  a 
variety  of  factors  and  probably  do  not  perfectly  measure  what  they  are  intended  to 
measure,  but  they  are  the  best  figures  available.  Additionally,  there  might  not  be  any 
correlation  between  the  quality  of  the  pilot  and  the  rank  of  graduation.  However, 
measuring  the  quality  of  the  pilot  would  be  extremely  difficult  because  it  is  subjective 
and  inherently  unknown  at  the  time  of  attendance  at  USAFA.  The  measurement  error 
observed  in  these  measures  of  performance  only  serves  to  enhance  the  results. 
Measurement  error  leads  towards  a  bias  to  zero  and  an  inflation  of  the  standard  errors. 
Both  of  these  serve  to  reduce  the  significance  of  the  results  observed.  If  more  precise 
measurements  were  available,  the  results  would  become  even  stronger  statistically. 

Another  potential  sticking  point  involves  the  timing  of  the  policy  change  taking 
effect,  versus  its  announcement.  Usually,  the  change  is  not  announced  until  shortly 
before  it  takes  effect.  This  means  that  the  people  facing  the  new  commitment  are  already 
at  the  Academy  and  owe  the  Air  Force  some  service  time.1  Had  they  had  perfect 

1  There  is  no  direct  monetary  cost,  such  as  tuition,  to  attend  the  Academy.  If  one  stays 
into  his  junior  year,  he  will  owe  the  Air  Force  some  amount  of  service  even  if  he  does  not 
graduate.  This  individual  can  leave  during  the  first  two  years  without  having  to  pay 
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information  and  known  the  commitment  length  they  would  be  facing  prior  to  entry  or 
reaching  the  point  of  owing  the  Air  Force  something  for  the  education  received,  they 
might  have  made  a  different  decision.  Since  they  cannot  undo  previous  decisions,  they 
ignore  the  change  because  the  additional  commitment  does  not  appear  as  great.  The 
policy  change  might  affect  people  who  are  considering  attending  the  Academy.  If  they 
alter  their  decision  based  on  the  new  commitment  length  and  do  not  attend  the  Academy, 
then  they  would  never  enter  the  dataset  and  thus  the  policy  change’s  impact  on  them 
would  not  appear  anywhere.  It  should  be  noted  that  this  will  lead  to  an  understatement  of 
the  results  found  in  this  thesis.  Taking  into  account  people  who  do  not  attend  USAFA 
because  of  the  policy  change  should  only  strengthen  these  results. 

An  additional  question  deals  with  the  relevance  of  the  state  macroeconomic 
variables.  Once  a  person  has  graduated  from  the  Academy,  the  Air  Force  sends  them  to 
where  they  are  needed.  If  the  decision  being  made  is  between  being  a  pilot  or  having 
some  other  career  in  the  Air  Force,  the  economic  conditions  in  the  home  state  may  not  be 
relevant  because  the  individual  will  not  be  returning  to  the  home  state.  However,  if  the 
decision  is  being  made  between  being  a  pilot  and  leaving  the  Air  Force,  then  the 
conditions  may  be  relevant. 

One  might  also  be  concerned  about  the  endogeneity  of  the  interaction  terms 
between  cadet  qualifications  and  the  policy.  The  policy  change  may  induce  some  cadets 
to  work  harder  to  secure  some  highly  competitive  non-pilot  positions  such  as  medical 
training.  However,  there  is  reason  to  believe  that  this  concern  is  not  a  large  one.  All  of 


anything  back.  After  that,  the  Air  Force  demands  that  he  serve  time  either  as  an  officer 
after  graduating  or  as  an  enlisted  troop  if  he  leaves  before  graduating. 


47 


the  two-way  interactions  are  re-estimated  with  all  qualifications  measures  divided  into 
quartiles.  The  results  are  found  to  be  substantively  and  statistically  almost  unchanged. 
Therefore,  the  results  herein  are  not  being  driven  by  subtle  changes  in  behavior  at  the  top 
of  the  qualification  distribution.  As  an  alternative  test,  the  top  and  bottom  10  percent  of 
the  qualification  distribution  are  eliminated.  While  the  precision  of  the  estimates  and  the 
magnitudes  of  the  effects  fall  when  this  exercise  is  conducted,  the  results  continue  to 
suggest  that  the  most  highly  qualified  cadets  remaining  in  the  sample  are  less  likely  to 
select  into  pilot  training  as  the  service  requirement  increases. 


CONCLUSIONS  AND  RECOMMENDATIONS 


This  thesis  shows  that  the  Air  Force’s  shift  in  policy  regarding  the  length  of 
commitment  owed  for  pilot  training  has  some  potentially  negative  unintended 
consequences.  The  increased  commitment  length  has  a  negative  impact  on  all  potential 
pilots.  Furthermore,  the  policy  change  has  a  stronger  effect  on  people  who  have  higher 
academic  and  overall  performance  scores.  This  means  that  there  is  a  potential  effect  of 
decreasing  the  quality  of  the  pilots  in  the  Air  Force.  While  the  policy  change  will  have 
its  intended  effect  of  addressing  the  problem  of  a  shrinking  pilot  force,  people  with 
higher  performance  measures  respond  to  it  more,  possibly  changing  their  decision 
regarding  becoming  pilots.  Additionally,  it  appears  that  the  effect  is  stronger  on  those 
individuals  who  have  shown  improvement  from  the  time  of  entering  the  Academy  to  the 
time  of  graduation  than  those  individuals  who  have  shown  a  decline  in  performance  over 
the  same  time  period.  This  too  would  seem  to  be  an  undesired  side  effect,  as  it  appears  to 
dissuade  the  type  of  people  who  would  be  preferred  more  than  those  who  might  be  less 
desirable.  The  policy  change  does  not  appear  to  have  significantly  different  effects  on 
different  demographic  groups  of  pilots.  Also,  the  local  macroeconomic  conditions  have  a 
questionable  effect  on  the  decision  to  attend  pilot  training. 

These  effects,  while  unintended,  might  be  acceptable  to  the  Air  Force.  All 
candidates  admitted  to  the  Air  Force  Academy  are  highly  capable  people  who  have  the 
potential  to  make  fine  officers  and  pilots.  The  policy  change  affects  only  these  capable 
people.  If  the  cadets  potentially  affected  by  the  policy  change  were  not  capable  of 
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becoming  quality  officers,  they  would  have  not  been  admitted  to  the  Academy.  Thus,  it 
can  be  argued  that  the  effect  is  perfectly  acceptable.  After  all,  it  is  not  the  situation  that 
the  people  becoming  pilots  now  are  drastically  lower  quality  than  those  who  became 
pilots  previously.  The  importance  of  all  the  effects  identified  and  discussed  in  this  thesis 
are  somewhat  subjective.  The  fact  that  they  are  observed  to  occur  does  not  show  that 
their  impact  is  extremely  detrimental  to  the  Academy  or  the  Air  Force.  There  is  no  study 
of  the  correlation  between  the  quality  of  a  pilot  and  performance  while  at  the  Academy. 
However,  it  seems  logical  to  think  that,  if  the  Academy  is  structured  properly,  there 
would  be  some  correlation.  It  is  possible  that  the  completion  of  the  Academy  program  is 
sufficient  demonstration  of  ability  and  the  actual  relative  performance  at  the  Academy  is 
not  relevant.  These  are  all  factors  that  must  be  considered  in  evaluating  the  overall  effect 
of  the  policy  change. 

The  point  of  this  thesis  is  to  demonstrate  that  there  is  no  free  lunch,  not  even  for 
the  Air  Force.  The  demonstration  of  this  fact  does  not  mean  that  the  policy  should  revert 
to  its  previous  form.  That  is  a  decision  that  must  be  made  by  the  Air  Force.  If  they  value 
the  increased  amount  of  service  of  pilots  in  the  future  due  to  the  longer  commitment,  then 
the  policy  is  acceptable.  However,  if  it  is  deemed  that  the  lowering  of  quality  is 
unacceptable,  than  another  solution  needs  to  be  explored. 
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